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Abstract 

We tested the natural combination of surrogate data analysis with 
the ApEn regularity statistic developed by Pincus [Proc. Natl. Acad. 
Sci. USA 88 (1991) 2297] by applying it to some popular models of 
nonlinear dynamics and publicly available experimental time series. 
We found that this easily implemented combination provided a use- 
ful method for discriminating signals governed by nonlinear dynamics 
from those governed by linear dynamics and noise. An apparently 
novel physical interpretation of ApEn also is supplied. 



1 Introduction 

Significant developments in the last decade have provided techniques that can 
discriminate between signals generated by essentially linear dynamics and sig- 
nals generated by essentially nonhnear and possibly chaotic dynamics[Tj [SI |1! • 
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Our goal here is to assess the utihty of the combination of surrogate data 
techniques with the "ApEn" (or "approximate entropy") statistic for the 
identification of essentially nonlinear signals. In the next two subsections 
we briefly recapitulate surrogate data methodology and the ApEn statistic, 
respectively. We present the results of applying the combined analysis to 
various data sets and models of nonlinear dynamics in the next Section. We 
conclude in Section 3 with some comments on the results. Three short ap- 
pendices follow that set forth some details of the algorithm for calculating 
ApEn, the connection between ApEn and standard quantities, and the pre- 
viously unnoticed numerical coincidence of ApEn with the thermodynamic 
entropy of the one-dimensional Ising model, respectively. 



1.1 Surrogate Data Methodology 

The idea of surrogate data is that one tests some hypothesis concerning a 
time series by comparing the value of some statistic Q for the original time 
series with the sample mean of that statistic Q in an ensemble of random 
replicates (the surrogates) generated from the original time series under that 
hypothesis. The hypothesis would be taken to be true if the original time 
series were statistically indistinguishable from the ensemble of surrogate time 
series. "The use of surrogate data is essential for deciding whether an irreg- 
ular time series arises from nonlinear deterministic chaos or linear stochastic 
dynamics. The method was introduced by Theiler et al. (1992)." [3] The 
particular method that we are concerned with was designated FT by Theiler 
et al. [ZI |H] , and tests the following linearity hypothesis: The original time 
series is the result of linear stationary dynamics driven by gaussian white 
noise, i.e., the data can be modeled by ARMA dynamicsjSI. The surrogates 
are produced by first randomizing the phase of the Fourier transform of the 
original time series, then inverting back to the time domain to create a new 
time series in the ensemble of surrogate time series. This procedure ensures 
that each surrogate time series possesses the same power spectrum as the 
original data. Theiler et al. suggested that the "significance", i.e., the num- 
ber S of standard deviations (Tq of Q between Q and Q, written as 

sJ^, (1) 

could be employed as a simple albeit crude test for the success or violation 
of the linearity hypothesis. They also suggested that the statistic Q could be 
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the correlation integral or correlation dimension, suggestions that have been 
widely adopted, although other statistics also have been suggested for the 
surrogate data methodp] [TU] fH] P^ij ^13ij . In view of the usual uncertainties 
in the estimate of and the crudeness of 5* itself in this work we required 
S* > 10 to score a violation, although other authors JH] have been willing to 
declare a violation of the linearity hypothesis with a significance as small as 
three. The analysis is meaningful only for stationary time series; examples of 
violations of the linearity hypothesis by linear non-stationary processes have 
been documented[14j. 

Several authors have proposed modifications to the FT method for gener- 
ating surrogate series. Theiler et al. themselves introduced the "amplitude- 
adjusted" method (designated AAFT by them^) because the amplitudes of 
the surrogate data generated by the FT method can vary significantly from 
the envelope of the original time series. It turned out that the AAFT it- 
self introduced other concerns that in turn have been addressed by yet more 
elaborate and in some cases more computationally demanding surrogate data 
methods!^ |15jP^ . Those proposals to modify the FT method retained the 
original suggestions for the statistics; here we pursue the opposite approach, 
retaining FT here for computational and conceptual simplicity, and employ- 
ing ApEn as an alternative statistic. Of course ApEn could have been just 
as easily employed with any of the other methods to generate a surrogate 
series. 

1.2 ApEn: A regularity statistic 

In this paper we address only the choice of the statistic in the FT method. 
One of our concerns with both the correlation integral and the correlation di- 
mension as a statistic is that each may require a large number of data points. 
The data required may grow rapidly with the correlation dimension, so that 
some workers have had to employ this or related statistics in a regime where 
the validity of doing so was difficult to assess [T7] [THI. One of the appeals of 
ApEn is that it has performed well, in other contexts [TO] pil] ^T] ^H] pi] . 
even on decimal data sets as small as N ?a 100, or binary data sets as small 
as N ^ 20. Therefore we investigated the application of ApEn to the FT 
surrogate data method, which we designate ApEn+FT. 

The statistic ApEn was developed by Pincus to measure the conditional 
probability that "...runs of patterns that are close for [some number] of ob- 
servations remain close..." p!U]. ApEn therefore depends upon the length [N) 
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of the series, the width (m) of the window that defines the patterns , and the 
tolerance (r) that defines closeness of the patterns, and incidentally provides 
ApEn with some resistance to noise. Pincus' algorithm for the calculation 
of ApEn is recapitulated in the Appendix A for reference. The theoretical 
(large N, m and small r) bounds to ApEn are zero for a perfectly regular 
sequence, and ln{B) for a maximally irregular (i.e., random) sequence of base 
B numbers. It turns out (see Appendix C) that ApEn numerically coincides 
with the thermodynamic entropy for the one- dimensional Ising model when 
each configuration of spins is interpreted as a string of bits. This coinci- 
dence serves at least to provide a physical interpretation of ApEn (for binary 
data), even though ApEn was constructed without any explicit reference to 
statistical mechanics or thermodynamics. However, we did not further ex- 
ploit this connection. Furthermore, in spite of its name and its asymptotic 
behavior (see Appendix B), we did not regard ApEn as an approximation 
to the, e.g., Kolmogorov-Sinai or Eckmann-Ruelle, non-thermodynamic en- 
tropies because the amount of data required to achieve useful approximations 
may be very large in some circumstances [T7] |18j |19j. and because those en- 
tropies can be especially sensitive to noise. We also did not compare ApEn 
with either the correlation integral or correlation dimension for three reasons. 
First, we have not attempted to establish the superiority of ApEn over all 
other statistics in all circumstances. Second, both the correlation integral 
and the correlation dimension nominally require the evaluation of limits that 
in turn may also require much larger quantities of data than ApEn itself|19j 
(see also Appendix B). Third, the comparison between ApEn and the cor- 
relation integral (for example) is complicated by the fact that the choice of 
parameters m and r that are optimal for one are almost certainly not optimal 
for the other; employing only the parameters that we also employed for the 
ApEn calculation might have appeared to be creating a "straw man" . 

Therefore, we adopted the alternative recommended by PincusjjST], i.e., 
that ApEn is simply a regularity statistic (or family of statistics) defined by 
the choice of its (A^, m, r) parameters that stands on its own. As such, ApEn 
has been applied to binary sequences j21j and to physiological dataj^H] |2S] |2I] ■ 
Although ApEn is a biased statistic, as are many nonlinear statistics, we 
always employed it to compare only the sequences with the same set of 
[N, m, r) parameters, so that the bias would not become an issue here. ApEn 
is strongly sensitive to the complexity of the data, but is insensitive to other 
attributes, as would be desired of such a statistic. For example, and not 
surprisingly, ApEn is insensitive to topological conjugation j2] of sequences. 
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Furthermore, ApEn is semi-pivotal in the sense that it is always insensitive 
to the sample mean of the data, and it is insensitive to the sample variance if, 
as Pincus suggests[19j, the tolerance parameter r is always chosen as a fixed 
fraction of the sample standard deviation. This feature contributed to the 
utility of ApEn in surrogate data methodologyiS^ ; in particular, the concern 
that inspired the AAFT surrogate data method should be insignificant here. 

Even though the combination of ApEn with surrogate data methodology 
is a natural one to employ, we found no examples of such analysis in the 
physical science or engineering literature. However we recently discovered 
two examples of this kind of analysis in the cardiology literature |26j |27j . 
although those applications appeared to us to be narrow, with no attempt to 
test the method with standard data or models, or to explore its limitations. 
Therefore we proceed to show the results of our tests of ApEn+FT in the 
next section. 

2 Results of tests with ApEn+FT 

The plan of this section consists of the employment and empirical testing of 
the ApEn statistic in the FT surrogate data method (ApEn+FT) to both 
experimental and computational data. In order to obtain consistent results, 
we followed published recommendations (1 9 j ^21^ to fix the pattern window 
length to m = 2 and the tolerance to r = 0.2a, where a was the sample 
standard deviation estimated by the non-parametric bootstrapP^. Although 
important questions remain concerning the optimal values of A^, m, and 
r, we make no pretense of answering them here but defer them to future 
work. However, preliminary estimates (for decimal numbers) suggest that 
the minimum number of points for an adequate calculation of ApEn requires 
on average only about SO'" points|2I]. Therefore we always considered data 
sets at least as large as = 2048. On the other hand, we did not examine 
data sets larger than N = 8192 (A^ = 4096 for most data sets), because 
we wanted to see if the ApEn+FT analysis would be useful for medium- 
size data sets; for very large data sets it is conceivable that any number of 
methods would work equally well, but such large data sets are rarely achieved 
experimentally. We also restricted A^ to powers of two so that we could use 
the simplest version of the Fast Fourier Transform|29j without having to pad 
the data. 

For each original time series we generated an ensemble of only ten FT 
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surrogate time series, because the fluctuation of ApEn in the ensemble was 
consistently small and because occasional tests with 40 surrogates did not 
yield significantly different estimates. As discussed below, the length of 
the series had a much greater impact on the performance of the hypothesis 
test than did the size of the surrogate ensemble. The percentile bootstrap 
method was employed to estimate the (0.5%, 99. 5%) or (0.1%, 99. 9%) confi- 
dence intervals employed in the Figures in order to assist visual inspection, 
but only the bootstrap estimate of the standard deviation was employed in 
the significance calculations (see Equation [T))[28j. The bootstrap usually pro- 
vides a good estimate of the sample standard deviation even for ten samples, 
especially where, as was the case here, the median and the mean are nearly 
coincident. 

We did not display the time series themselves because we found that it was 
misleading to visually compare the original time series with its surrogates. 
The surrogate data do not need to overlap point for point with the original 
in order for them to be statistically the same. The FT method randomizes 
the phase, but ApEn is properly insensitive to the phase (if any) present 
in the data, so that even when the original and surrogates have the same 
ApEn, the time series themselves may appear to be quite different from 
one another. On the other hand, some series (e.g., Henon) vary so rapidly, 
that the original and surrogates may appear similar when they are actually 
statistically distinct. 

2.1 Noise 

We began by studying the behavior of ApEn for white noise, in order to 
provide a context for the values of ApEn for the subsequent data, and to 
motivate the combination of ApEn with the surrogate data techniques. We 
generated an ensemble of ten time series from a white noise source pro- 
vided here by two uniform pseudorandom number generators, respectively: 
a standard multiplicative-congruential generator, implemented as "ran2" [21] , 
and a much different generator employing a random-walk algorithm imple- 
mented as "rawkrab" 'HU' . Each generator gave the same result for the sample 
mean, i.e., ApEn(A^ = 4096, m = 2, r = 0.2a) = 2.11 (with a standard de- 
viation estimated to be 0.01), about 9% below the theoretical maximum 
ApEn(A^ — > oo,m —>■ cxD,r ^ 0) = In(lO) JHl- We continued this calibration 
by generating another ensemble of ten time series directly from the Brownian 
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noise (or Ornstein-Uhlenbeck) power spectrum |31j 



for each dissipation rate 7, i.e., we assigned random phases to 5*0 and com- 
puted its inverse Fourier transform to obtain each trajectory]^. Such tra- 
jectories automatically satisfy the linearity hypothesis. The resulting time 
series is a solution of a Langevin equation, which can be represented simply 
for discrete times by the ARMA model[33.] x„ = Cq + Cix„_i + C2^„, with 
time-invariant coefficients Cj(A^, 7), and the uncorrelated gaussian noise of 
zero mean and unit variance C,n- Figure ^ shows the sample mean ApEn 
computed for each of the various 7; the standard deviation (estimated by 
bootstrap techniques |2H1) is about 0.01, or about 1/10 the diameter of the 
circles. For small 7, the spectrum becomes a smeared delta function, the 
trajectories become more regular, and ApEn is near zero. For larger 7, 
the spectrum becomes flatter, the trajectories become more random, and 
ApEn approaches its white noise limit (i.e., 2.11) for the parameters em- 
ployed above. For intermediate values of 7 the trajectories appear noisy but 
more regular, as shown in the inset of Figure Q for 7 = 1/ 64. Thus any 
allowed value for ApEn can be found from the Brownian time series. There- 
fore, instead of relying upon the value of ApEn alone, we combined it with 
surrogate data generation in order to test the hypothesis of linear dynamics. 



2.2 Data sets and models 

We applied ApEn+FT to data sets and models that have been widely dis- 
cussed and for which the time series can be either readily computed or freely 
downloaded, as follows: the data sets labeled "A" (chaotic laser), "D" (turbu- 
lent flow), and "E" (light variability from a star) were taken from the Santa Fe 
Institute's 1991-1992 competition jSl] (but not the other data sets that 
were represented as significantly non-stationary); an experimental realiza- 
tion of the Chua circuit [37j ; the x-coordinate of the Lorenz system jlj; 
the x-coordinate of the Henon mapj2]; the Mackey-Glass model and the 
x-coordinate of the two-dimensional map of the non-chaotic strange attractor 
of Grebogi, Ott, Pelikan, and YorkejSHl- In all we obtained 23 original time 
series, for which we first computed ApEn, as shown in Figure |21 For each of 
these series we also generated an ensemble of ten FT surrogate time series. 
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Figure 1: ApEn for white and colored noise. Each circle corresponds to 
sample mean of ApEn computed for an ensemble of ten trajectories generated 
(via uniformly random phases) from the Brownian noise spectrum S'o(ci;,7) 
(Equation©), for various values of 7. The standard deviation is estimated 
to be 0.01, about 1/10 the diameter of the circles. The inset shows a typical 
time series reconstructed from 5*0(0;, 7 = 1/64). For reference, the dashed 
line shows the value of ApEn uniformly distributed (pseudo)random numbers 
on the interval [0, 1]. The dotted line shows the value of In(lO), the ApEn for 
uniform random decimal numbers in the limit of large N and m and small 
r. The circle at the origin is theoretical, not computed. 
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and calculated the sample-mean ApEn for each ensemble. We calculated 
(0.5%, 99.5%) confidence intervals about by the non-parametric percentile 
bootstrap method||2Hl jSHl- The results are displayed in Figure El and Table [T] 



Table 1: Significance of ApEn+FT test 



Data Set 




S 


A (mean) 


4096 


120 


D (mean) 


4096 


50 


E (all) 


2048 


< 5 


chua 


4096 


30 


lorenz 


8192 


300 


henon 


4096 


200 


mackey-glass 


4096 


5 


mackey-glass 


8192 


25 


strange 


4096 


230 



Both visual inspection of Figure and the estimation (see Equation 
H} of the significance S (Table H} indicated that the series labeled A, D, 
chua, lorenz, henon, mackey-glass, and strange clearly violate the linearity 
hypothesis, as expected. To put these results into perspective, others 
have been willing to declare a violation of the linearity hypothesis with a 
significance as small as 3 (using other statistics), while we insisted upon a 
significance of at least 10 (see Equation The small magnitude of the 
fiuctuations of ApEn for the surrogate data contributes substantially to the 
large values of 5* in the case of violations of the linearity hypothesis. The 
Mackey-Glass system is the smoothest and most nearly periodic of any of the 
time series that we considered here and was also one of the most resistant 
to our analysis. The time series of 4096 points (not shown in Figure EI) 
resulted in a significance well below our criterion of S" > 10. In order to 
find a more significant violation of the linearity hypothesis, we generated the 
8192-point time series (see Table Q). Again, the fiuctuations in the ApEn of 
the surrogate data were small. On the other hand, the Lorenz system did 
not need N = 8192 for the violation to become apparent with this analysis; 
we presented the results in order to indicate the extreme significance that 
could be achieved with this analysis. 
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Figure 2: ApEn of original and surrogate time series for various data sets 
and models. The shading of the columns is used here to group data sets. The 
top of each column shows the ApEn of the original time series. The height of 
each pair of error bars shows the (0.5%, 99.5%) bootstrap confidence interval 
about sample mean (not shown) of the ApEn of the surrogate time series. 
The labels "A", "D", and "E" correspond to the Santa Fe Institute 1991- 
1992 Competition Data Sets Arcontinuation, D and E, respectively. Multiple 
columns labeled A and D correspond to subsets split in increments of 4096 
points from the original data set. The columns labeled E correspond to parts 
8, 11, 13, and 14, respectively, of Data Set E, and contain only the first 2048 
measurements of each part. The column labeled "chua" corresponds to the 
first 4096 points of a relatively noiseless 5000-point time series of the voltage 
from an experimentally realized network of two Chua circuits. The next two 
columns correspond to = 8192 time series generated numerically by us for 
the x-coordinate of the Lorenz system (Pr = 16, Ra = 45.92, and 6 = 6) 
and the x-coordinate of the N — 4OS0 Henon map (a = 1.4 and b — 0.3), 
respectively. The next column corresponds to the N — 8192 time series 
generated numerically by us for the Mackey-Glass model. The last column 
corresponds to the = 4096 time series generated numerically by us for the 
x-coordinate of the two-dimensional non-chaotic strange attractor (A = 1.5) 
of Grebogi et al. 



The ApEn+FT analysis indicated that series E (hght variabihty of a star) 
did satisfy the hnearity hypothesis, as suggested by both visual inspection 
of Figure El and estimation of the significance (Table Q). If E were actually 
governed by nonlinear dynamics, then this would have been a failure of our 
analysis. Our difficulty with the Mackey-Glass system also suggests that 
our analysis might have failed, because at = 2048, the series in E are 
the shortest that we have considered. On the other hand, inspection of the 
data sets themselves and their power spectra suggested that the fluctuations 
in E might be more random than recent and more substantiated examples 
of nonlinear light variability of astronomical objectsjlO] [H]. Therefore we 
were unable to decide between the conclusion that the data in E really are 
nonlinear but are obscured (to our analysis) by noise, or that the data in E 
really are governed by ARMA dynamics, as indicated by our analysis. 



2.3 Sensitivity to noise 

In order to gain a sense of the magnitude of the noise needed to disrupt 
our analysis, we added uncorrelated gaussian noise ^{t) (zero mean, unit 
variance), scaled by a factor, to the experimental Chua circuit and the Henon 
map. We chose these time series for comparison because the Chua system 
violated the linearity hypothesis with relatively small significance {S ~ 30) 
while the Henon map had a much larger significance {S ~ 200). By trial 
and error we estimated the scale factor such that the addition of (pi(t) 
to the time series reduced S to ten. In Table El we show the range (i.e., 
distance between the extreme values) of the original time series, (p, and p, 
the corresponding signal-to-noise ratio |29j: 

^ Y.t{signal{t)Y 
^ Y.t{noise{t)Y ■ ^ ' 



Table 2: Magnitude of the noise required to reduce the significance 
ApEn-hFT test to ten 



Data Set 


Range 


<P 


P 


chua 


6 


0.35 


40 


henon 


3 


0.45 


2.4 
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The ApEn+FT analysis of the experimental Chua circuit began to fail 
with the addition of as little as 2^% noise, while the artificial Henon map 
could withstand up to 42% noise. However, we did not find enough compara- 
ble discussions of the signal-to-noise to make comparisons to other methods, 
so we could not decide from results like these whether our method was su- 
perior or inferior to other methods with respect to noise sensitivity. 

2.4 Henon map subject to a linear filter 

Experimental data is sometimes filtered even before it becomes "raw" dataj42j. 
Two examples were considered for evenly-sampled discrete data x„: the mov- 
ing average (MA) filter in Equation ^ , and the linear autoregressive (AR) 
filter in Equation El 

1 ^ 

~ 9^11 ^rn+n (4) 

d x^—i ~1~ x^ (5) 

We wanted to assess the impact of these filters on data that were clearly 
governed by nonlinear dynamics. Therefore we needed data that appeared 
noisy or jerky because the filters would have little effect on smooth data, 
(e.g., the Mackey-Glass model) even if they were nonlinear. We chose the 
data from the Henon map as described in the subsection above, because of 
its apparent susceptibility to filtering, and applied both filters, for various 
values of the filter parameters q and a, respectively. The results are listed in 
Table El The original series (see Figure |2l) has an ApEn of 0.48. The sample 
mean ApEn of the surrogates of the filtered data differed very little from 
that of the surrogates of the unfiltered data. However, the results in Table O 
indicate that filtering increased the ApEn of the original data by as much as 
50%, but the linearity hypothesis still was significantly violated {S > 100). 

2.5 Henon map tested with AAFT surrogate data 

We have said throughout that we did not regard as crucial the particular 
form of the method employed to generate the surrogate time series. Never- 
theless, we observed that the amplitudes of the FT surrogate data for the 
Henon map were nearly twice that of the original data (in each direction), 
the most we observed for any of the data sets, so we thought it prudent to 
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Table 3: ApEn for MA- and AR- filtered Henon map. ApEn = 0.48 for the 
unfiltered Henon time series. 



q (MA) 


ApEn 


a (AR) 


ApEn 


3 


0.69 


0.1 


0.46 


7 


0.76 


0.3 


0.43 


15 


0.65 


0.5 


0.51 


31 


0.59 


0.7 


0.69 


63 


0.58 


0.9 


0.62 



check the consequences of such a difference in the amplitudes by applying 
the amplitude- adjusted (AAFT) method [B| to this data. Although the AAFT 
surrogates look much more like the original data, the corresponding sample 
mean ApEn was found to be 1.94, only about 5% lower than the sample 
mean ApEn (2.03) of the FT surrogate data; the sample standard devia- 
tions, estimated to be 0.006 and 0.008 for the AAFT and the FT surrogates, 
respectively, also differed little from one another. 



2.6 Logistic map with varying parameter 

The results above show the behavior of the ApEn statistic with FT surrogate 
data analysis for a wide range of real data and models. However, none of 
these provided insight into the behavior of the analysis as the regularity of 
the data varies. Here we considered the logistic map in order to examine the 
behavior of the analysis with a parameter that changes the regularity of the 
data. 

The logistic map is usually defined by[T] 

Vn+l = fiVnil-yn) (6) 

or equivalently, with a different choice of coordinates, j2] 

Xn+l = 1 — p x"^ 

. ^ f (f - 1) (T) 

Figure El shows the value of ApEn of the time series generated from Equation 
ini and its surrogates for various values of p. For fi < 3.55, the dynamics 
are consistent with the linearity hypothesis. ApEn begins to rise quickly 
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in the interval 3.55 < fj. < 3.70, that contains the Feigenbaum attractor 
(/i ~ 3.57) j2], although the comparison with the surrogates showed that 
the linearity hypothesis still passed for 3.55 < fi < 3.58. In the interval 
3.58 < fi < 3.60 the linearity hypothesis finally began to be violated, and 
at /X = 3.60, the linearity hypothesis was violated with S > 50. Thus this 
analysis identified the critical parameter to within one percent of the correct 
value. 

2.7 Kaplan- Yorke map with varying parameter 

The ApEn+FT analysis identified the transition between linear (i.e., peri- 
odic) and nonlinear behavior with the variation of a single parameter in the 
logistic map. We next examined the transition between chaotic to random 
behavior with one of the simplest of the Kaplan- Yorke family of maps^, 
which is a two-dimensional extension of the logistic map for /x = 4 (see 
Equation [Tj) : 

yn+i = kyn + Xn (8) 

This map has the following physical interpretation [2]: the ^/-coordinate 
describes the velocity of a unit mass particle in a dissipative medium sub- 
ject to kicks of strength x„ at times nr. For a fixed dissipation rate 7, 
the dissipation parameter k = exp(— 7r). In the small-r limit, i/r^/n de- 
scribes a gaussian random process, with integral information dimension^. 
For moderate r (i.e., < |), however. Equation |H1 describes complex chaotic 
dynamics, with a fractional information dimension. Figure lU shows ApEn for 
the component of Equation |H1 and its surrogates, for various k. Although 
the ApEn of the original series of yn is fiat for all but the highest values of k, 
the distance between the ApEn of the original and the surrogate data begins 
to smoothly decrease for k > ^. The significance is at least 100 for k < 0.95. 
Finally, at k = 0.99, the ApEn of the original and the ApEn of the surrogate 
data begin to coincide {S ~ 8), consistent with an ARMA process. It is 
interesting that the value of ApEn alone did not indicate the transition from 
nonlinear to linear dynamics, since ApEn actually decreased for the random 
case. Instead, the transition became apparent only in conjunction with the 
surrogate data analysis. 
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Figure 3: ApEn and FT surrogates for the logistic map for various coupling 
constants. The circles and squares represent ApEn for the original and the 

surrogate series, respectively. The error bars represent the bootstrap estimate 
of the (0.5%, 99.5%) confidence interval for the surrogate data. 
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Figure 4: ApEn and surrogates for the Kaplan- Yorke map for various cou- 
pling constants. The solid circles show the ApEn of the original time se- 
ries. The height of the error bars represents the bootstrap estimate of the 
(0.1%, 99.9%) confidence interval for the surrogate data. The dotted hne (fit 
by a sphnc) represents the difference between the ApEn of the original data 
and the mean ApEn of the surrogate data. 
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3 Comments and Conclusion 



The success of ApEn+FT analysis in detecting the nonhnearity in the noise- 
less chaotic models (i.e., Lorenz, Henon, logistic, Kaplan- Yorke, Mackey- 
Glass) was gratifying, but was neither unique nor unexpected; indeed such 
success was necessary for further investigation. The example of the Mackey- 
Glass model also showed that the results could be quite sensitive to the 
length of the original time series. The ability of this analysis to distinguish 
between the complex chaotic behavior and the gaussian random behavior 
of the Kaplan- Yorke model, although not unique, adds to our confidence 
in the method. It was also encouraging to see that even relatively severe 
smoothing of the Henon series was not sufficient to obscure its inherent non- 
linearity. The ability of the ApEn+FT to detect nonlinear dynamics in the 
Santa Fe Institute Data Sets A (chaotic laser) and D (turbulence) data sets, 
although also not unexpected, was a more useful test because those data sets 
contained some noise, and in the increments considered here, were of inter- 
mediate length. The success of the linearity hypothesis for Data Set E with 
ApEn+FT was inconclusive, because this result might have been an accurate 
assessment of that data, or it might have been a failure to detect nonlinear 
dynamics in a noisy series. The strong sensitivity of this analysis to the noise 
in the Chua circuit suggests that, as with all other techniques [2] |1] jS] , caution 
must be applied when the data is both noisy and short. Direct comparison 
to other methods was obstructed by the lack of equivalent analyses in the lit- 
erature, and performing such analyses ourselves seemed beyond the scope of 
this paper. Nevertheless, our goal was to empirically assess the performance 
of the ApEn+FT method, and this has been obtained with the results of the 
last section. 

The ApEn+FT method, like other implementations of surrogate data 
methods, cannot test for chaos, but only for the nonlinearity in the dynamics 
of a time series that, if present, merely indicates the potential for such a series 
also to be chaotic. The analysis can say nothing further concerning possible 
attractors, embedding dimension, or even if the data resulted from chaotic 
dynamics. Indeed one of the most spectacular violations of the linearity 
hypothesis came from the non-chaotic map of Grebogi, Ott, Pelikan and 
Yorke IMl- 

In addition to the problem of false positives (false success of the linearity 
hypothesis), there remains the possibility of false negatives (false violation of 
the linearity hypothesis) in ApEn+FT. For example, consider the sawtooth 
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series^P, with a spectrum that is poorly estimated by the straightforward 
Fast Fourier Transform employed in the FT method [21] . Each of the resulting 
surrogate time series would be much more irregular than the original time 
series, so that the ApEn of each would be significantly higher than the nearly 
zero ApEn of the original series, thereby yielding a false violation of the 
linearity hypothesis. However, to the extent that even complicated periodic 
systems might be easily recognized by visual inspection of either the series 
itself or its spectrum, the problem of false negatives might not pose the 
difficulties that false positives present. If more faithful surrogate time series 
were required, methods other than FT could be employed (see Introduction), 
and the ApEn method could be subsequently employed just as easily as we 
did here. 

In conclusion, the ApEn statistic is easily implemented in the surrogate 
data technique to test the hypothesis of linear dynamics. Violations of the 
hypothesis were found with large significance for a wide variety of both exper- 
imental and computer-generated time series that were known or suspected to 
be governed by nonlinear dynamics. The ApEn+FT method appears to be 
useful especially for discriminating between stochastic but linear and complex 
but nonlinear time series of moderate size if the noise is not too large. 
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A Algorithm for ApEn 

The algorithmic interpretation of ApEn, i.e., the conditional probability that 
"...runs of patterns that are close for [some number] of observations remain 
close..." was developed by Pincus as follows[19j. For a sequence {uk}, with 
< k < N, the N — m window vectors w{j) for < j < N — m, each of 
window length m, are defined by 

W{j) = [Uj,. . . ,Uj+rn]- (9) 

The fraction C^{r) of the distances from a given window vector w{i) to all 
the window vectors (including itself) that lie within a tolerance r is defined 
by 

where 6(x > 0) = 1 and 6(x < 0) = 0, and the distance d is given by the 
Li norm, i.e., 

d[w{i),w{j)] = niax \ui+k-Uj+k\- (H) 

0<A;<m 

Then "the CY^{r) measure, within a tolerance r, the regularity of patterns 
similar to a given pattern of window length m".[2I] Collecting these ideas, 
Pincus defined ApEn by 

ApEn(A, m,r) = <l>'"(r) - $™+^(r), (12) 

$-(r) = E ln(Cr(r)) 

N -m ^ 



B ApEn and related quantities in various lim- 
its 

There were at least two ways to employ ApEn to measure regularity in data, 
but only one of these was be fruitful for our application. The differences arise 
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from the way the parameters m, r, and employed. Viewed in the hmits 
— s> oo, m — s> oo, and r — 0, Pincus showed that the definitions above 
yield, at least theoretically|19.^, the more familiar correlation integral Tm{r) 
and correlation dimension (3 as follows: 

1 N-m-l 

r-f''* = i'SLN^ 2: Cr(r), (13) 

^ (14) 



m — *oo 



The theoretical bounds to ApEn in these limits are for a perfectly reg- 
ular sequence, and ln(i?) for a maximally irregular (random) sequence of 
base B numbers. ApEn itself theoretically converges to the Eckmann-Ruelle 
entropy [20] [22]- Pincus has explored the large- A^ limit numerically for bi- 
nary numbers, taking advantage of their special structure [221 • However, for 
most work with decimal numbers, the convergence is so slow that the limits 
above are practically unobtainable, therefore we did not attempt to employ 
them in this work. 



C Connection between ApEn of binary strings 
and the Ising model 

As we show in Figure ApEn, even with m = 2, essentially reproduces 
the thermodynamic entropy of a one-dimensional zero-field Ising model, a 
two-state spin model in which the ground state is anti- ferromagnetic, i.e., 
Ti-T-LTi • • with nearest-neighbor interactions on a periodic chain P] |I3]|44j . 
This coincidence seems not to have been noticed before, and because it pro- 
vides a physical analogy for ApEn, we digress here briefiy to include a few 
details of a calculation we carried out. Although the thermodynamics (in- 
cluding the entropy) are known exactly for this model[lH]; in order to apply 
ApEn, we generated the equilibrium configurations (a string of 1024 bits) 
from a Monte-Carlo simulation that employed importance sampling and pe- 
riodic boundary conditions^44j. For each temperature we generated a total of 
50 000 equilibrium configurations (binary strings), and computed the mean 
ApEn(A^ = 1024, m = 2, r < 1) from 20 configurations that were separated 
by 2000 passes, after discarding the first 10 000 configurations. The compar- 
ison of the mean ApEn with the exact thermodynamic entropy is displayed 
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in the Figure El We estimated the standard deviation of the mean ApEn to 
be between 0.001 and 0.002 for all temperatures. The high temperature limit 
corresponds to ln(2). There is a rough correspondence between the behavior 
of ApEn with varying temperature T for the Ising model and as a function 
of the dissipation rate 7 for the Brownian noise model (see Figure P). 
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Figure 5: ApEn and the exact entropy for equilibrium configurations of the 
Ising model for various inverse temperatures. The solid line is the exact 
thermodynamic entropy, and the circles are the calculated mean ApEn. The 
estimated standard deviations are much smaller than the circle diameter. 
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